Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tests: run uwsgi tests in virtual environment #191

Merged
merged 1 commit into from
Sep 4, 2023
Merged

tests: run uwsgi tests in virtual environment #191

merged 1 commit into from
Sep 4, 2023

Conversation

P403n1x87
Copy link
Owner

Description of the Change

We run the uwsgi tests in separated virtual environment to ensure that we install it using the interpreter that Austin is being tested against. Currently, we install the dependency in the environment used to run pytest, which means that we test uwsgi with the same Python version in every job.

Regressions

We make the uwsgi tests run as expected, so there is always the chance of new CI findings.

@P403n1x87 P403n1x87 self-assigned this Sep 4, 2023
@codecov
Copy link

codecov bot commented Sep 4, 2023 •
edited
Loading

Codecov Report

Patch coverage has no change and project coverage change: -0.77% ⚠️

Comparison is base (55f56fb) 68.04% compared to head (614ed45) 67.28%.

❗ Current head 614ed45 differs from pull request most recent head 7a628a9. Consider uploading reports for the commit 7a628a9 to get more accurate results

Additional details and impacted files 
@@            Coverage Diff             @@
##            devel     #191      +/-   ##
==========================================
- Coverage   68.04%   67.28%   -0.77%     
==========================================
  Files          27       27              
  Lines        2485     2485              
  Branches      761      761              
==========================================
- Hits         1691     1672      -19     
- Misses        458      471      +13     
- Partials      336      342       +6

see 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions
Copy link

github-actions bot commented Sep 4, 2023 •
edited
Loading

Austin Benchmarks

Running Austin benchmarks with Python 3.10.12

Benchmark Summary

Comparison of dev against 3.5.0.

The following scenarios show a statistically significant difference in performance between the two versions.

Sample Rate Saturation Error Rate Sampling Speed
RSA keygen [sampling interval: 100] 🟡 🟡 🟡 🔴
Full metrics [sampling interval: 100] 🟡 🟡 🟢 🟡
Full metrics [sampling interval: 1000] 🟡 🟡 🟡 🟢
Multiprocess wall time [sampling interval: 100] 🟡 🟡 🟡 🟢

Benchmark Results

Wall time [sampling interval: 1]

Wall time [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 118000 ± 3000 1 ± 0 1.3e-05 ± 9e-06 13.1 ± 0.3
3.5.0 121000 ± 1000 1 ± 0 1e-05 ± 4e-06 13 ± 0
dev 120000 ± 3000 1 ± 0 9e-06 ± 4e-06 13.1 ± 0.3
Wall time [sampling interval: 10]

Wall time [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 115000 ± 4000 0.543 ± 0.002 1.3e-05 ± 4e-06 13.3 ± 0.5
3.5.0 114000 ± 5000 0.545 ± 0.001 9e-06 ± 4e-06 13.8 ± 0.6
dev 116000 ± 6000 0.545 ± 0.0007 1.1e-05 ± 7e-06 13.4 ± 0.5
Wall time [sampling interval: 100]

Wall time [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 10200 ± 200 0.0011 ± 0.0009 3e-05 ± 3e-05 14.7 ± 0.5
3.5.0 10000 ± 400 0.0007 ± 0.0002 3e-05 ± 3e-05 14.4 ± 0.7
dev 10300 ± 300 0.0008 ± 0.0002 4e-05 ± 3e-05 14.1 ± 0.3
Wall time [sampling interval: 1000]

Wall time [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 1860 ± 20 0.0001 ± 0.0002 2e-05 ± 7e-05 18 ± 1
3.5.0 1850 ± 20 0.0 ± 0.0001 0.0001 ± 0.0001 17.1 ± 0.3
dev 1860 ± 20 0 ± 0 0.0001 ± 0.0001 16.9 ± 0.3
CPU time [sampling interval: 1]

CPU time [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 51000 ± 4000 1 ± 0 1.9e-05 ± 8e-06 23 ± 1
3.5.0 52000 ± 3000 1 ± 0 1.8e-05 ± 7e-06 22.5 ± 1.0
dev 54000 ± 2000 1 ± 0 1.5e-05 ± 7e-06 22.0 ± 0.7
CPU time [sampling interval: 10]

CPU time [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 54000 ± 2000 0.99795 ± 7e-05 1.3e-05 ± 6e-06 22.2 ± 0.4
3.5.0 53000 ± 2000 0.998 ± 0.0002 1.5e-05 ± 1e-05 22.3 ± 0.5
dev 53000 ± 2000 0.9979 ± 0.0006 1.1e-05 ± 1e-05 22.3 ± 0.8
CPU time [sampling interval: 100]

CPU time [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 5350 ± 90 0.0011 ± 0.0002 9e-05 ± 6e-05 22.8 ± 0.8
3.5.0 5400 ± 100 0.0012 ± 0.0003 0.0001 ± 0.0001 22.4 ± 0.5
dev 5400 ± 100 0.0012 ± 0.0003 0.00011 ± 9e-05 22.5 ± 0.7
CPU time [sampling interval: 1000]

CPU time [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 950 ± 1 0.0 ± 0.0001 0.0 ± 0.0001 26.1 ± 0.3
3.5.0 951 ± 2 0.0001 ± 0.0003 0.0 ± 0.0001 26.1 ± 0.9
dev 951 ± 2 0 ± 0 0.0002 ± 0.0002 25.9 ± 0.3
RSA keygen [sampling interval: 1]

RSA keygen [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 39000 ± 1000 1 ± 0 0.0005 ± 0.0006 25 ± 1
3.5.0 39700 ± 700 1 ± 0 0.0003 ± 0.0002 24.4 ± 0.7
dev 39000 ± 1000 1 ± 0 0.0002 ± 0.0002 24.6 ± 0.8
RSA keygen [sampling interval: 10]

RSA keygen [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 39500 ± 400 0.98 ± 0.01 0.0002 ± 0.0002 24.9 ± 0.3
3.5.0 40000 ± 1000 0.98 ± 0.02 0.0002 ± 0.0001 24.1 ± 0.9
dev 40000 ± 1000 0.98 ± 0.01 0.0002 ± 0.0001 23.8 ± 0.6
RSA keygen [sampling interval: 100]

RSA keygen [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 6230 ± 70 0.003 ± 0.003 0.0008 ± 0.0007 30 ± 2
3.5.0 6250 ± 30 0.0018 ± 0.0006 0.0004 ± 0.0003 29 ± 2
dev 6240 ± 40 0.002 ± 0.002 0.0004 ± 0.0008 31 ± 2
RSA keygen [sampling interval: 1000]

RSA keygen [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 943 ± 1 0 ± 0 0.001 ± 0.002 32 ± 2
3.5.0 943.2 ± 0.9 0 ± 0 0.001 ± 0.002 32 ± 3
dev 940 ± 5 0 ± 0 0.0 ± 0.001 31 ± 1
Full metrics [sampling interval: 1]

Full metrics [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 50000 ± 3000 1 ± 0 3e-05 ± 1e-05 31 ± 2
3.5.0 51900 ± 800 1 ± 0 3e-05 ± 1e-05 29.6 ± 0.5
dev 52000 ± 1000 1 ± 0 3e-05 ± 1e-05 29.6 ± 0.8
Full metrics [sampling interval: 10]

Full metrics [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 52000 ± 2000 1 ± 0 3.2e-05 ± 1e-05 30 ± 2
3.5.0 52000 ± 2000 1 ± 0 2e-05 ± 1e-05 30 ± 2
dev 52000 ± 1000 1 ± 0 2e-05 ± 1e-05 29.3 ± 0.9
Full metrics [sampling interval: 100]

Full metrics [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 10000 ± 300 0.0018 ± 0.0004 8e-05 ± 7e-05 33 ± 1
3.5.0 10000 ± 200 0.0018 ± 0.0004 8e-05 ± 5e-05 33 ± 1
dev 9900 ± 200 0.002 ± 0.001 1e-05 ± 2e-05 32.6 ± 0.7
Full metrics [sampling interval: 1000]

Full metrics [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 1870 ± 20 0.0003 ± 0.0003 0.0001 ± 0.0001 37.3 ± 0.8
3.5.0 1860 ± 20 0.0002 ± 0.0003 0.0001 ± 0.0002 37.3 ± 0.7
dev 1870 ± 10 0.0002 ± 0.0002 0.0001 ± 0.0002 36.5 ± 0.7
Multiprocess wall time [sampling interval: 1]

Multiprocess wall time [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 3100 ± 200 1 ± 0 0.00017 ± 4e-05 300 ± 20
3.5.0 3400 ± 200 1 ± 0 0.00016 ± 3e-05 280 ± 10
dev 3400 ± 200 1 ± 0 0.00014 ± 4e-05 280 ± 10
Multiprocess wall time [sampling interval: 10]

Multiprocess wall time [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 3000 ± 200 1 ± 0 0.00016 ± 3e-05 310 ± 20
3.5.0 3300 ± 100 1 ± 0 0.00015 ± 4e-05 290 ± 10
dev 3320 ± 100 1 ± 0 0.00015 ± 4e-05 288 ± 9
Multiprocess wall time [sampling interval: 100]

Multiprocess wall time [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 2800 ± 200 0.05 ± 0.02 9e-05 ± 4e-05 330 ± 30
3.5.0 3300 ± 100 0.05 ± 0.01 0.0001 ± 0.0002 290 ± 10
dev 3400 ± 100 0.05 ± 0.01 5e-05 ± 3e-05 278 ± 8
Multiprocess wall time [sampling interval: 1000]

Multiprocess wall time [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.4.1 2700 ± 100 0.014 ± 0.004 1e-05 ± 2e-05 110 ± 10
3.5.0 3100 ± 100 0.007 ± 0.002 1e-05 ± 2e-05 65 ± 7
dev 3100 ± 200 0.007 ± 0.001 1e-05 ± 1e-05 64 ± 5

@P403n1x87 P403n1x87 force-pushed the tests/uwsgi-venv branch 4 times, most recently from a323058 to 614ed45 Compare September 4, 2023 10:45
@P403n1x87 P403n1x87 marked this pull request as ready for review September 4, 2023 10:46
@P403n1x87
tests: run uwsgi tests in virtual environment
7a628a9 
We run the uwsgi tests in separated virtual environment to ensure that
we install it using the interpreter that Austin is being tested against.
Currently, we install the dependency in the environment used to run
pytest, which means that we test uwsgi with the same Python version in
every job.
@P403n1x87 P403n1x87 merged commit 7815662 into devel Sep 4, 2023
29 of 36 checks passed
@P403n1x87 P403n1x87 deleted the tests/uwsgi-venv branch September 4, 2023 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@P403n1x87 P403n1x87

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@P403n1x87